Skip to content

Simplify jemalloc setup #146627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Simplify jemalloc setup #146627

wants to merge 2 commits into from

Conversation

@madsmtm
Copy link
Contributor

@madsmtm madsmtm commented Sep 16, 2025 •
edited
Loading

In the past, #[used] had to appear in the top-level crate to have a consistent effect on the linker. This has been fixed a while ago for ELF with the introduction of the symbols.o file in #95604, and more recently for Mach-O in #133832, which means that libraries can now implement the required workarounds themselves. This allows moving these #[used] declarations out of our main.rs.

Specifically, I have moved them into tikv-jemalloc-sys where they belong in tikv/jemallocator#109 and done the same for mimalloc in purpleprotocol/mimalloc_rust#146 (in case we want to experiment with switching to that one day).

Test with:

./x build library src/tools/rustdoc src/tools/clippy --set rust.jemalloc=true

# macOS
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b _rjem_je_zone_register
(lldb) run
# Should breakpoint, this means that the allocator was properly linked

# Linux
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b malloc
(lldb) run
# Should breakpoint, inspect that the `malloc` symbol comes from the `rustc` binary and not from `libc`

try-job: aarch64-gnu
try-job: dist-aarch64-linux
try-job: dist-x86_64-musl
try-job: dist-x86_64-apple
try-job: dist-aarch64-apple

@madsmtm madsmtm added A-linkage Area: linking into static, shared libraries and binaries T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Sep 16, 2025
@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-clippy Relevant to the Clippy team. S-blocked Status: Blocked on something else such as an RFC or other implementation work. labels Sep 16, 2025
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@madsmtm madsmtm added O-linux Operating system: Linux O-macos Operating system: macOS labels Sep 17, 2025
@madsmtm
Copy link
Contributor Author

madsmtm commented Oct 25, 2025 •
edited
Loading

tikv-jemalloc-sys v0.6.1 has been released with the mentioned PR, so this is now ready. I'm less certain about the Linux parts here, so please double-check that.

Also, in the second commit, I fixed librustdoc and clippy building with --features jemalloc / ./x build --set rust.jemalloc=true, unclear to me how this wasn't caught by CI? Are these built differently there?

@rustbot ready
r? compiler
@bors try

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-blocked Status: Blocked on something else such as an RFC or other implementation work. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 25, 2025
rust-bors bot added a commit that referenced this pull request Oct 25, 2025
@rust-bors
Auto merge of #146627 - madsmtm:jemalloc-simplify, r=<try>
8d39231 
Simplify `jemalloc` setup

try-job: `aarch64-gnu`
try-job: `dist-aarch64-linux`
try-job: `dist-x86_64-musl`
try-job: `dist-x86_64-apple`
try-job: `dist-aarch64-apple`
@rust-bors

This comment has been minimized.

Sign in to view
@madsmtm madsmtm marked this pull request as ready for review October 25, 2025 21:52
@rustbot
Copy link
Collaborator

rustbot commented Oct 25, 2025

Some changes occurred in src/tools/clippy

cc @rust-lang/clippy

The Miri subtree was changed

cc @rust-lang/miri

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

@madsmtm madsmtm added T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. A-allocators Area: Custom and system allocators and removed T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Oct 25, 2025
@rust-bors
Copy link

rust-bors bot commented Oct 26, 2025

☀️ Try build successful (CI)
Build commit: 8d39231 (8d39231d22563844d4088a7af200b6885c749c85, parent: 79966ae420f38c5861d177356a3446023c090d6d)

RalfJung
RalfJung reviewed Oct 26, 2025
View reviewed changes
@madsmtm
Simplify jemalloc setup
7a9cafe 
Using the new `override_allocator_on_supported_platforms` feature in
`tikv-jemalloc-sys v0.6.1` we can avoid the manual statics.
@madsmtm
Fix building rustdoc and clippy with jemalloc feature
2c0d266
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@RalfJung RalfJung RalfJung left review comments

Assignees

@jdonszelmann jdonszelmann

Labels

A-allocators Area: Custom and system allocators A-linkage Area: linking into static, shared libraries and binaries O-linux Operating system: Linux O-macos Operating system: macOS S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-clippy Relevant to the Clippy team. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@madsmtm @rust-log-analyzer @rustbot @RalfJung @jdonszelmann